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Abstract — Due to its numerous advantages, orthogonal fre- 
quency division multiplexing (OFDM) has been the broadband 
wireless access technology of choice for many wireless standards 
over the last decade. Recently, filter bank based multicarrier 
techniques are emerging as one of the alternatives to OFDM 
for next generation broadband wireless access systems. While 
there are several studies on filter bank based multicarrier systems 
in the literature, its potential merits over OFDM and possible 
implementation challenges in practical channel and impairment 
conditions require further investigations. The goals of the present 
survey are to provide an extensive and unified review of waveform 
design issues for filter bank based multicarrier systems, shed 
some light on advantages and disadvantages of this technology, 
and provide pointers to open research problems. 

Index Terms — biorthogonal modulation, filter-bank multicar- 
rier (FBMC), future radio access, hermite pulse, IOTA, OFDMA, 
BFDM, PHYDYAS filter, polyphase network (PPN), pulse shap- 
ing, waveform design 



I. Introduction 

Increasing number of users, wide variety of wireless com- 
munication applications, and extensive growth of the user 
demands from the wireless communication systems lead more 
adaptive, flexible and efficient future radio access techniques. 
Over last decade, orthogonal frequency division multiplexing 
(OFDM) has been the radio access technology of choice for 
many wireless standards, such as 802.1 la/b/g under WiFi 
alliance, 802.16 (WiMax), DVB-T/DAB-T, WRAN 802.22, 
and Long Term Evolution (LTE) of cellular radio. Some 
advantages of OFDM include simple equalization in frequency 
domain, less complex implementations through fast Fourier 
transformation (FFT), efficient usage of spectrum through 
frequency overlapping while limiting time signal over a perfect 
rectangular block, and suitability for multiple-input multiple- 
output (MIMO) systems. In addition, possibility to exploit 
multiuser diversity and frequency diversity make OFDM an 
appealing technology for many wireless systems. On the other 
hand, new waveform design for the next-generation broadband 
wireless access (BWA) systems carries critical importance due 
to shortcomings of OFDM such as high sidelobes, suscepti- 
bility to carrier frequency offset (CFO) and doubly dispersive 
channels, cyclic prefix (CP) overhead, and larger peak- to- 
average-power ratio (PAPR). 

Filter bank multicarrier (FBMC) is one of the BWA options 
that may address the shortcomings of OFDM while maintain- 
ing some of its advantages. Earlier works related to FBMC 
actually date back to 1960s (TJ, E3, which utilize a bank of 
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Fig. 1 : Transmitter and receiver block diagrams in OFDM and 
FBMC. 



filters for parallel data transmission. In JT|, Chang presented 
the orthogonality condition for the multicarrier transmission 
schemes considering band-limited filters and offset quadrature 
amplitude modulation (OQAM). This condition basically in- 
dicates that the subchannels can be spaced half of the symbol 
rate apart without any inter-carrier interference (ICI) and inter- 
symbol interference (ISI) when the data is staggered on alter- 
nate in-phase and quadrature subchannels. This multicarrier 
transmission idea has then been generalized by Saltzberg in 
1967 [2 by showing the fact that Chang's condition is also 
true when the time and frequency axes are interchanged. 
However, the idea of parallel transmission suggested in JT| 
and J2] required unreasonably expensive and complex for large 
number of data channels at that time. In [3 ], through the use of 
discrete Fourier transformations (DFTs), Weinstein and Ebert 
eliminated the banks of subcarrier oscillators and allow simpler 
implementation of the multicarrier schemes. This approach has 
been later named as OFDM, and it has become more and more 
popular after 1980s due to its efficient implementation through 
FFT techniques and frequency domain equalization (FDE) 
along with CP utilization [4] compared to FBMC approached 
On the other hand, Weinstein 's DFT method in limits 
the flexibility on different baseband filter utilization while 
modulating or demodulating the subcarriers, but instead used 
a time windowing technique to cope with the ICI. In 01, 
by extending Weinstein 's method, Hirosaki showed that the 

the literature, the phrase of orthogonal frequency division multiplexing 
is conventionally used for the scheme that is based on rectangular filters. How- 
ever, FBMC approaches with other filters can also satisfy the orthogonality 
conditions. Indeed, the phrase of orthogonal frequency division multiplexing 
embraces the FBMC scheme. 
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different baseband filter may also be digitally implemented 
through DFT processing by using a poly phase network 
(PPN) O, Q. As shown in Fig. [U transmitter/receiver block 
diagrams of OFDM and FBMC are very similar to each other. 
At the transmitter, inverse fast Fourier transformation (IFFT) 
operation is followed by CP insertion in OFDM (to combat 
with ISI), while it is followed by a PPN in FBMC. In the 
same manner, CP removal is substituted with PPN in FBMC 
to extract the information symbols at the receiver. Several other 
developments over the last two decades have demonstrated low 
complexity and efficient implementations of FBMC, paving 
the way for its consideration in the next generation wireless 
standards (see e.g., lt8ll- lfTll . and the references listed therein). 

In the literature, two main advantages of FBMC compared 
to the conventional OFDM are emphasized. First, FBMC may 
use any prototype filter design (captured through the PPN). 
By applying different filter coefficients to PPN structure, it is 
possible to yield better adaptations to channel characteristics 
and other design constraints without changing the imple- 
mentation. Secondly, FBMC exploits the sparser structure of 
time-frequency grid compared to the conventional OFDM. 
Therefore, it allows the use of better-localized prototype 
filters in both time and frequency domains. For example, the 
lattice grid with staggered modulations in FBMC effectively 
halve the density of the lattice with quadrature amplitude 
modulation (QAM) El, E2- Therefore, FBMC provides a 
way to improve the ISI/ICI performances of the systems in 
doubly dispersive channel with the proper filter selections. 
In the literature, this interference immunity is exploited to 
remove the guard time between symbols, which yields more 
spectral efficient schemes compared to OFDM. Beside these 
well-known advantages, if the concept of biorthogonality, i.e., 
different prototype filters at the transmitter and receiver, is 
included to FBMC, it is also possible to show the impact 
of prototype filters to equalization domain. For example, CP- 
OFDM offers FDE by using inherently unsymmetrical utiliza- 
tion of the filters, i.e., rectangular filter at the receiver and the 
extended rectangular pulse at the transmitter, which obeys to 
the nature of biorthogonality lH3l . (14]. With this perspective, 
a generic framework to study different waveforms including 
OFDM and equalization approaches can be constructed in the 
sense of different configurations of the filters. 

With the recent developments in filter bank based ap- 
proaches and the urge to improve next generation wireless 
access capacity and flexibility dictated by the wireless traffic 
trends fT31 . filter bank based multicarrier systems emerges as 
an umbrella for the transmission schemes, and promises to 
be an alternative radio access technology. The goal of this 
survey paper is to provide an extensive and unified review 
of waveform design issues for filter bank based multicarrier 
systems. Pointing open research problems on FBMC also 
constitutes another major aim of this survey paper by shedding 
some light on advantages and disadvantages of the filters. 

The paper is organized as follows: The classification of 
multicarrier systems is provided considering both orthogo- 
nal and biorthogonal Gabor systems in Section [II] Then, 
well-known prototype filters in the literature are represented 
mathematically and the trade-offs between these filters are 




k + 1 



Sampling the 

time-frequency plane 

-r- t , with the lattice 

Time-frequency plane 

(a) Gabor system at the transmitter. Prototype filter is employed in time-frequency 
plane at different locations considering the lattice at the transmitter to construct 
the transmitted signal. The product of vqtq corresponds to the sampling density 
of the lattice. Another Gabor system is also constructed at the receiver to recover 
the informations symbols. 
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(b) Considering parallel transmission, projection of the received signal on to the 
analysis function corresponds to getting sample after received filter every other 
ro and vq. 

Fig. 2: Utilizing the prototype filters at the transmitter and the 
receiver sides in time and frequency domains in FBMC based 
systems. 



explained in Section Jill Useful tools and metrics to eval- 
uate the filter performances are investigated in Section JV] 
Then, the comparison of the filters are provided in view of 
time-frequency characteristics, sidelobe performance, signal- 
to-interference ratio (SIR) performance in wireless channels 
and truncation in Section [V] Considering all of these inves- 
tigation on filters, transceiver design issues on FBMC are 
investigated from different perspectives in Section |VT] Finally, 
the paper is concluded in Section IVII1 

II. Multicarrier Communications 

In a multicarrier transmission, the information symbols are 
transmitted over the points of a lattice as follows [16], fTTvTl 



(1) 



where p tx (t) is the prototype filter, is is the subcarrier 
spacing, ro is the symbol spacing, m is the symbol index, and 
k is the subcarrier index. The family of functions in (H) is often 
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referred to as a Gabor system, and p m k (t) is called in various 
names like synthesis function, window, atom, or pulse shape 
which maps information symbols into time-frequency plane 
considering a lattice grid for different m and k as in Fig. [2 |[a)| 
lfT8l . Therefore, considering TV subchannels, the transmitted 
signal in the baseband is given by lfT2l . |fT9l 

oo N-l 

X(t) = ^ X ™kPmk(t) , (2) 

m— — oo k=0 

where X m h is the complex or real information symbol trans- 
mitted at the grid point (m,k). Then, the spectral efficiency 
of this system can be written as lfT9l , |[20l 

r] = — [bits/sec/Hz] , (3) 

where f3 is the number of bits per symbol. 

After the transmitted signal given in ® passes through 
the linear time- varying wireless channel h(r,t), the received 
signal is obtained as 

y(t)= J h(r,t)x(t-r)dr + w(t) 

= J J H(r,v)x(t - r)e j27rut dudr + w(t) , (4) 

where H{t,v) is the Fourier transform of h(r,t) and w(t) 
is the additive white Gaussian noise (AWGN). Therefore, as 
indicated in Fig.[ 3fb)[ the received symbol X n \ located on time 
index n and frequency index I is obtained by the projection 
of the received signal on to analysis function p n \ (t) as 

Xnl = (y(t),Pnl(t))± jy(t)p* nl (t)dt 

— X n iH n i n i + y ^ XmhHnimk + w n i , (5) 
(m,fe)#(n,i) 

where 

Pni(t)=Pr X (t-nr )e^ l ^ t (6) 

is the receiver filter offset in time by nro and in frequency by 
Ivo, w n i is the noise after projection, and 

H n lmk= I I H{T,V) [prnkit-T^ty^dtdvdT 

(J) 

captures the interference from the symbol (m, k) to the desired 
symbol (n, I). Similar to ©, p n i(t) given in © is obtained 
by another prototype filter p rx (£) shifted in both time and 
frequency domains depending on (n, I). In other words, both 
Pmk(t) and p n i(t) denote banks of filters based on Gabor 
system at the transmitter and receiver sides. 

A. Reconstruction and Representation in Gabor Systems 

The time-frequency characteristics of p tx (t) and p rx (t) 
employed in the scheme, and the lattice structure identified 
by ro and v$ are related with the representation and the 
reconstruction properties of FBMC based transmissions. These 
properties can be investigated via Gabor analysis. In Gabor 
analysis, the product of vqtq indicates the density of the 



sampling lattice as in Fig. [2 Ha)[ It determines not only the 
spectral efficiency of the transmission as in d3}, but also the 
representability of the signal space, and the invertibility of the 
transformation between information symbols and constructed 
signal in © under AWGN or ideal channels. Basically, the 
perfect reconstruction of a signal compels to use linearly 
independent set of p m k(t), since this property corresponds to 
invertibility of a transformation. On the other hand, the rep- 
resentability of the Hilbert space £ 2 (R) of square-integrable 
functions with the set of p m k(t) is equivalent to the com- 
pleteness property of the Gabor system. While including an 
extra basis function to the basis spoils the linear independency, 
discarding a basis function from the space destroys the com- 
pleteness of the system. However, it is important to note that 
whereas linear independence is conservative condition for the 
communication systems, completeness is not that important 
ll22l . As an extension of the concept of a basis, Frames, 
introduced in 1952 by Duffin and Schaeffer [23], might not 
be linearly independent. In other word, a frame might include 
more than the required functions to span a space. This issue 
yields to an overcomplete systems, which might cause lossy 
reconstruction [24], [|25l . Considering the spectral efficiency, 
linear independency and completeness of the systems, FBMC- 
based systems are classified for an ideal channel regarding to 
the product of uqTo in TABLE U and described as follows lfT6ll . 
EQ: 

• Undersampled grid (Vo^o > 1): In this case, Gabor 
system given in (QJ cannot be a complete basis or a frame 
since the lattice is not sampled sufficiently. However, it 
still allows linear independency between the basis func- 
tions. Therefore, well-localized prototype filters obtained 
from incomplete bases can be adapted to the communi- 
cation systems. However, the spectral efficiency of the 
system degrades with increasing voTo product when the 
complex information symbols are utilized. On the other 
hand, transmitting in-phase and quadrature components 
in staggered manner, e.g., OQAM, the spectral efficiency 
can be still maintained as voTo = 1. Also, this approach 
yields an undersampled grid structure where vqTq = 2. 

• Critically sampled grid (Vo^o = 1): It results in a com- 
plete Gabor System. Orthogonal bases exist, but accord- 
ing to the Bailan-Low theorem [26], critically sampled 
grid cannot utilize well-localized prototype filters, and 
instead, dictates the filter with bad time-frequency local- 
ization, e.g., rectangular filter. The spectral efficiency is 
maximized for to^o = 1, since the perfect reconstruction 
might not be possible beyond the critically sampled grid 
condition [16]. Pure OFDM symbols, which does not 
to use CP or zero padding (ZP), is constructed with 
critically sampled grid. However, employing CP or ZP 
leads a system with undersampled grid because of the 
coarsification of the lattice in time domain. 

• Oversampled grid (VoTo < 1): In this case, an overcom- 
plete set of functions is obtained. Gabor system cannot be 
a basis, but it might be a frame with well-localized pulse 
shape. However, since the Gabor system is overcomplete, 
in a perfect channel, representation of a signal might not 
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Perfect Reconstruction 



Conditional Perfect Reconstruction 



v t > 1 

(e.g. SMT, CMT, FMT, CP/ZP-OFDM) 



(e.g. pure OFDM) 



v t < 1 



Linear Independent Systems (Bases) 



Linear Dependent Systems (Frames) 



Incomplete Systems for L 



Complete Systems for L 



Overcomplete Systems for L 



Orthonormal Systems (e.g. pure OFDM, ZP-OFDM, SMT, CMT, FMT) 

Xnl — C^ XmkPmk >Pnl) an ^ Pmk = Pmk 
\mk I 



Biorthogonal Systems (e.g. CP-OFDM) 

Xnl — \ ) t XmkPmk >Pnl) = ^mkPmk > Pnl J an ^ Pmk ^ Pmk 
\mk I \mk ' 



TABLE I: Classification of FBMC-based schemes respect to the product of voTo in Gabor systems. Selecting voto > 1 
allows linear independent bases which are crucial for the communication systems. While the orthogonal systems dictate to 
use the same set of functions at the transmitter and receiver, biorthogonal system allows to use different Gabor systems at the 
receiver and the transmitter with the perfect reconstruction property. Employing different Gabor systems at the transmitter and 
receiver introduces a flexibility on selecting the basis to combat with the doubly dispersive channels. On the other hand, the 
reconstruction of information symbols perfectly in Gabor systems is conditionally possible when v>oto < 1 because of including 
more than required number of basis functions. For example, employing finite- alphabet for the transmitted information symbols 
can allow a system where vqTq < 1 with the perfect reconstruction property [21]. 



be unique. Therefore, the reconstruction might be lossy 

d, G3. 

When voTo < 1, specific conditions might yield the perfect 
symbol reconstruction. This issue firstly is investigated by 
Mazo in 1974 as faster- than Nyquist (FTN) by addressing the 
following question: what extend can be the symbols packed 
more than the Nyquist rate without loss in bit error rate (BER) 
performance? It is shown that the symbol spacing can be 
reduced to 0.802T without suffering any loss in minimum 
Euclidean distance for binary information symbols and sine 
pulse 1 27]. In other words, ISI-free BER performance is still 
achievable with the optimal maximum a posteriori (MAP) 
sequence detector even when the symbols are transmitted at a 
rate greater than Nyquist rate. The minimum symbol spacing 
that keeps the minimum Euclidean distance is later on called 
Mazo limit. By generalizing FTN approach to other pulses, 
various Mazo limits are obtained for root-raised cosine (RRC) 
pulses with different roll-off factors in [28 1. For example, 
when roll-off is set to 0, 0.1, 0.2, and, 0.3, Mazo limits are 
derived as 0.802, 0.779, 0.738, and, 0.703 respectively. The 
FTN approach is extended to multicarrier schemes by allowing 
both ICI and ISI in O, f30ll . It is shown that two dimensional 
signaling is more bandwidth efficient than one dimensional. 
Another way of developing a scheme where voTo < 1 is 
to transmit correlated symbols. This approach corresponds to 
partial-response signaling (PRS) and introduced in [31|. Simi- 
larly, employing finite- alphabet for the transmitted information 
symbols along with frames can allow a system where isqTo < 1 
with the perfect reconstruction property [21]. 

In this survey, we consider only complete and incomplete 
linear independent systems, i.e. vqTq > 1, to discuss the 
schemes allowing the practical communication systems. 



B. Orthogonal and Biorthogonal Systems 

For the sake of clarity, if the channel is assumed ideal 
and the noise is ignored in ©, the information symbols are 
obtained by 

Transmitted signal 



X n i = ( ^ X rnk p rnk (t) , p n i (t) 



(8) 



Received symbol 



In finite dimensional vector space, this operation corresponds 
to 



Transmitted signal 

, ^ s 

X-rnk ~ T T ^mk 

V v ' 

Received symbol 



(9) 



where 



T= 



Po,o 
Po,i 

Po,N 



, T -1 = 



P0,N 



(10) 



For an orthogonal basis, T _1 is equal to T H where [] H is 
the Hermitian operator. However, the equality of T H and T _1 
dictates to use the same Gabor systems at the transmitter and 
receiver, and it corresponds to choosing p t x (t) = p rx (t). 

Extending the concept of orthogonal basis, a collection of 
basis functions is called as Riesz basis if it is possible to 
generate a basis via converting the orthogonal basis func- 
tions with an invertible linear transformation. Therefore, even 
though the obtained basis is not constructed with orthogonal 
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set of functions, i.e, T _1 ^ T H , it is possible to obtain 
another basis where the inner product of the functions selected 
in those sets yields to TT _1 = /. In other words, perfect 
reconstruction is still possible by employing two different 
Gabor systems where p tx (t) ^ p rx (t) at the transmitter and 
the receiver. In that case, the Gabor systems, i.e., T and T _1 , 
constitutes a biorthogonal system. Since p m k (t) and p n \ (t) are 
interchangeable they are called as dual bases. 

A nice interpretation on orthogonality and biorthogonality is 
provided in ll32l . If an operator, i.e. known as frame operator, 
is defined as S = TT H , the reconstruction of X m k is given 
by 

SS X-mk S ^ S S ^ ^ 

S- p TT H S- {1 - p) X mk 

Transmitted signal 



X 



mk 



= S~ P T x (S- {1 - p ^T) H X. 



mk 



(ID 



Received symbol 

where p is the weighting parameter between orthogonality and 
biorthogonality. While the choice of p —> 1/2 yields a more 
orthogonal system, p — >> or p — > 1 increase the biorthogo- 
nality. Note that using orthogonal basis functions, similar to 
the match filtering, maximizes signal-to-noise ratio (SNR) for 
AWGN channel. On the contrary, the biorthogonality offers 
better performance in a dispersive channel. Therefore, it is 
possible to change the weight on orthogonality-biorthogonality 
via p depending on the dispersiveness of the channel ll32ll . 

C. Classification of Multicarrier Systems 

Considering the theory of Gabor systems, various FBMC 
schemes are investigated depending on to, vq, and the infor- 
mation symbols. If complex information symbols are used, it 
is possible to synthesize three main schemes with perfect re- 
construction property; CP/ZP-OFDM and FMT are introduced 
in Fig. [ 3Ia)| OFDM without any guard time is a canonical 
example when tqVq — 1. However, as indicated before, 
the well-localized filter cannot be utilized. Thus, employed 
filters make the system susceptible to the dispersions in the 
channel. Therefore, ISI is typically handled through the use 
of a CP. This prevents interference in time domain, at the 
cost of reduction of spectral efficiency to /3/(F(T + Top)) 
by selecting uq = F = 1/T and to = T + Xcp, where 
Xcp is a cyclic prefix duration to combat with multipath. 
On the contrary, FMT, which has been proposed before for 
digital subscriber lines (DSL) applications in |[33l and fl34l , 
separates the subcarriers by selecting > F instead of 
overlapping adjacent subcarriers. Therefore, FMT can provide 
same lattice sampling density of CP/ZP-OFDM. However, the 
guard interval in OFDM and guard bandwidth in FMT do 
not address the time/frequency spreading in the same manner. 
While CP-OFDM provides time spreading control along with 
FDE, FMT does not offer the same degree of freedom for 
frequency spreading. FMT rather comes into prominence for 
controlling the out-of-band leakage. Illustrations of OFDM and 
FMT are given in Fig. [4] 

One of the key goals in FBMC systems to control the inter- 
ferences from other lattice grid points, i.e. ISI and ICI in time 
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(a) Multicarrier schemes employing in-phase and quadrature components over 
the same lattice. By extending voto, the lattices for OFDM with CP or ZP, 
and FMT-based FBMC can be obtained. The room for the time-frequency 
localization of the prototype filter depends on uoro. While extending voro 
relaxes the design of the prototype filter, it also degrades the spectral 
efficiency of the system as in OFDM with CP or ZP and FMT-based FBMC. 
( n : Symbols on both quadrature and in-phase components) 
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(b) Multicarrier schemes employing in-phase and quadrature components 
over the staggered lattices. All even- symmetric and real prototype filters 
provides orthogonality among the symbols located on multiple of 
distances away for in-phase and quadrature components separately. Therefore, 
the room for the time-frequency localization of the prototype filter is relaxed 
compared to the schemes that place in-phase and quadrature component over 
the same lattice. By aligning the places of the symbols, different schemes, 
i.e, SMT, CMT, and IMT, can be obtained. ( • , A : Symbols on quadrature 
component, O , A : Symbols on in-phase component) 

Fig. 3: Representations of various multicarrier schemes on 
time/frequency plane. 



and frequency domains, to a certain symbol while maintaining 
the maximum spectral efficiency. As opposed to extending the 
lattice, this can be achieved through well-designed prototype 
filters with using staggered lattices for in-phase and quadrature 
components. Four different symbol shapes are indicated in 
Fig. [3 [Ib)| As long as the prototype filter is an even and real 
function, it is guaranteed that there will be no interference 
among the symbols located on multiples of distances 
away for in-phase and quadrature components separately (see 
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(d) CMT. 

Fig. 4: Illustrations of various multicarrier schemes in time 
and frequency domains. While OFDM provides the orthogo- 
nality by overlapping the sinc-type of functions in frequency 
domain, FMT provides the orthogonality without overlapping 
in frequency domain. Both FMT and OFDM can provide the 
same lattice density with guard band and guard time. On the 
other hand, SMT and CMT exploits the staggered structures 
to achieve flexibility on filter design while maintaining the 
spectral efficiency of tqUq = 1. CMT and SMT can be 
obtained by exchanging the time and frequency axes. 



e.g. [1T21 for proof). On the other hand, the interference among 
the other points of the same shape is avoided/minimized 
through a well-designed prototype filter. A basic comparison 
on non- staggered and staggered structures can be given as 
follows: 

Observation-I: The lattice density of in-phase component 
or quadrature component in a staggered structure is equal to 
the half of the lattice density of non-staggered one. Therefore, 
staggered structures yield more room for prototype filter 
design. 

Observation-II: The lattice density of a non-staggered struc- 
ture is equal to the half of the lattice density of staggered one. 



However, since complex symbols are transmitted at each grid 
point in non-staggered structures, both modulation formats 
have the same spectral efficiency in the absence of a guard 
interval or band. 

According to Observation-I, having more room for the 
filter design in staggered structures allows ISI and ICI to be 
controlled via well-designed filters. In other words, staggered 
structures introduce the paradigm of the localizations of the 
ICI and the ISI with the proper filters while maintaining 
spectral efficiency. Thus, they indirectly might allow the 
practical equalizers which exploit the localized ICI and ISI to 
combat with doubly dispersive channels. However, since there 
will be intrinsic interference between quadrature and in-phase 
components in a multipath channel, computational complexity 
becomes the bottleneck for the staggered structures. 

Staggering in-phase and quadrature lattices leads two im- 
portant modes used often in the literature, i.e. SMT and CMT 
ifTTl . These two modes are very similar to each other as the 
points on the lattice have the same product of tqVq as given 
in Fig. 1 3[b)[ The illustrations for SMT and CMT are given in 
Fig. |U Indeed, by aligning the position of the pulses in the 
lattice, it is possible to obtain other modes. For example, a 
mode which is called IMT in this study can be synthesized by 
setting as r = V2T/2 and u Q = V2F/2. 

In the literature, FBMC approaches are often related with 
the orthogonal transmission and reception. For example, 
OFDM without CP, ZP-OFDM which provides a guard period 
between each OFDM symbols, and FMT, SMT, and CMT 
based FBMC schemes are based on the orthogonality of the 
basis functions. However, FBMC can also be extended to 
biorthogonal schemes which utilize different filters at the 
transmitter and the receivers [|T6l . Note that orthogonality 
is not a requirement for perfect construction, however it 
provides minimum error caused by AWGN [|32l . On the 
other hand, biorthogonality is an important fact that can yield 
better immunity to doubly dispersive channels compared to 
the orthogonal systems |[T6l . Also, it might allow additional 
relaxations on the equalization complexity. Using an extended 
rectangular filter at the transmitter and removing the extended 
part with another rectangular filter as in CP-OFDM is the most 
prominent example of biorthogonal systems [|T3l , f[T4l . One 
can interpret that CP-OFDM benefits from the biorthogonal 
filter utilization which allows FDE by providing circularity 
in time domain. While an orthogonal system results in better 
SIR performance for voro > 1 compared to V0T0 = 1, a 
biorthogonal system with optimal pulses can minimize the 
interference contribution from other information symbols in 
doubly dispersive channel by allowing different filters at the 
transmitter and the receiver. Indeed, optimizing the filters 
subject to the less interference contribution do not consider 
the equalization complexity. Thus, instead of optimizing the 
filters for SIR performance solely in doubly dispersive chan- 
nel, biorthogonal schemes with suboptimum filters can lead 
simpler equalization, e.g., FDE as in CP-OFDM. 

Single-carrier transmissions which allows better PAPR char- 
acteristics and FDE along with CP utilization as in single 
carrier frequency domain equalization (SC-FDE) and single 
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carrier frequency division multiple accessing (SC-FDMA) 
schemes Il35l - l39l can also be explained with FBMC ap- 
proaches if one consider an additional filter bank that spreads 
the energy of the information symbols in frequency domain. 
In other words, as oppose to use single Gabor system at 
the transmitter and receiver, two Gabor systems combined 
with serial-to-paralel conversion at the transmitter and the 
receiver are employed to yield single carrier systems. For 
example, SC-FDMA employes filter banks constructed with 
a rectangular filter to spread (i.e., DFT) and de- spread (i.e., 
inverse DFT) the information symbols in frequency domain 
at the transmitter and receiver. On the contrary, in OFDM, 
since there is no effort on spreading the information symbols 
in frequency domain, employed prototype filter spreading in 
frequency domain corresponds to Dirac function. Therefore, 
considering spreading the information symbols in both time 
and frequency and biorthogonal/orthogonal schemes, filter 
bank spread FBMC (FB-S-FBMC), which is only investigated 
for orthogonal systems in [40], provides a broader transmission 
and reception framework from mathematical perspective. 

III. Prototype Filter Design 

FBMC allows flexibility on the design of prototype fil- 
ters considering orthogonal and biorthogonal schemes when 
voTo > 1. This flexibility motivates the prototype filter design 
for different purposes. Goal of this section is to review them, 
and discuss their trade-offs. The analytical expressions of all 
filter considered are given in TABLE HH Then, the filters are 
investigated by classifying them under three main branches 
considering their theoretical definitions: time-limited filters, 
band-limited filters, and localized filters. 

A. Time -limited Filters 

Time-limited filters consist of the filters where their theo- 
retical definitions are given for finite durations as the name 
implies. The spectrum of these filters have infinite length 
due to the finite durations. Since they are specifically design 
for finite duration, they provide easy implementations of the 
systems. 

1) Rectangular Filter: Rectangular prototype filter is de- 
fined by L12J. It is the prototype filter used in OFDM. 
If the rectangular filter is extended by Xcp, conventional 
CP utilization is obtained. Also, it is possible to control 
the spectral leakage of the rectangular filter by introducing 
additional windowing periods fiTTl . Extending the rectangular 
filter allows multipath delay spread immunity. However, this 
approach is likely to be a suboptimal solution under doubly 
dispersive channels, since this approach does not treat the time 
and frequency dispersions equally [|T6l . 

2) Hamming, Hanning and Blackman Filters: In [11], 
Hamming, Hanning, and Blackman windows (see also (42J, 
R31 ) are specified with K\, if 2, ^3 respectively denoting the 
normalizing coefficients to satisfy the following constraint 

\j\m = i, o<t<T , (12) 

[p(i) = , t < and t > T . 

where Tq is the duration of the filter. 



3) Optimal Finite Duration Pulses/Prolate: In FBMC sys- 
tems, filters have to be truncated because of the computational 
complexity which is directly proportional to the filter length. 
However, using shorter or truncated filter causes undesired 
sidelobes in frequency domain as in rectangular and truncated 
RRC filters. As a tempting solution for this trade-off, achieving 
a time-limited pulse with minimum sidelobe power constitutes 
the main motivation of prolate spheroidal wave functions 
(PSWFs). The set of prolate functions can span the Hilbert 
space £ 2 (R) of square-integrable functions. In other words, 
PSWF constructs a complete set of functions. In order to 
achieve better understanding of the PSWFs, first, discrete 
prolate sequences, which are also called as Slepian sequences 
have to be investigated. Two main features of these sequences 
are distinctive compared to other basis; 

• These sequences are orthonormal for a given sequence 
length. This is also true for fundamental basis, i.e., sine 
and cosine as in Fourier transformation between — 2tt. 
However, the sequence length is an input parameter for 
Slepian sequences. This is important since the length of 
the sequences corresponds to the truncation or a finite 
duration pulses in communication systems. 

• These sequence are eigenfunctions of a perfect low pass 
filter. Therefore, these sequences can pass through a 
low pass filter without any distortion or filtering effect 
excluding the scaling with a real coefficient, i.e, eigenval- 
ues, which corresponds to the energy in that bandwidth. 
Assuming that the energy of the pulse is 1, eigenvalues 
will always be less than 1. 

These two features mean that if one provides the filter length 
and the bandwidth where the pulse has to fit as the design 
constraints, the optimum pulse becomes the function which 
has the maximum eigenvalue in the set constructed with 
these constraints. It is worth to note that since the functions 
are orthogonal to each other, the strongest eigenf unction is 
favorable compared to the others. This issue is explained as a 
sidelobe minimization problem in ifTTI as 

f roc r Af/2 } 

p(£) = argmhJ / |P(/)| 2 d/- / \P(f)\ 2 df\ . 

Pit) yj-00 J-Af/2 J 

(13) 

However, even if the sequence with maximum eigenvalue is 
optimum in terms of sidelobe for a given filter length, it does 
not solely satisfy the Nyquist criterion ensuring zero ICI and 
ISI. Considering this fact, Vahlin exploits PSWFs to realize 
a new family which is called as optimal finite duration pulse 
(OFDP) 1441 by generalizing the optimization procedure for 
single carrier presented in [45]. Basically, the aim is to achieve 
a Nyquist filter in both time and frequency with the maximum 
energy in the main lobe for a given bandwidth and filter 
length. In order to develop these pulses, Vahlin chooses the 
signal representation as the linear combinations of the PSWFs. 
Indeed, another basis can be selected to construct these pulses. 
However, the required number of basis function might be 
very high. Vahlin formulates the constraints as an optimization 
problem to find the weights (ak) for kth PSWF (^fc, prolate (t)) 
truncated to interval [—(K/2)T, [K/2)T] using Lagrange mul- 
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Family 



TABLE II: Analytical expressions of well-known prototype filters in the literature. 
Filter Analytical Model 



Parameters 



Time-limited 



Rectangular 



? lct ^ , otherwise 



Ext. rectangular 



Perec (t) 



Hamming 



Hanning 



Blackman 



OFDP/Prolate 



Kaiser 



PHYDYAS 



2VT + 2VT cos ^ T W ) 

Vt ' 
1 

, 



- ? - T C p — T w <£<—? — T C p 



1 cos (-^) 



< t < ^- + Tw 
otherwise 



Phamm (*) = ^1 (0.54 - 0.46 COs( ) j n f 



*-T /2 



(*) = ^2 (0.5 - 0.5cos(^))n(^fZ^] 



Pblackman (*) = K 3 (0.42 - Q.5 COs( ^) + 0.08 COs( gj ) n ( ) 



, prolate 

(*) 

fc = 



Pka 



(t)= (/o(/3vT^(2n/7V)2)// (/3) , -f < n < 



AT 
2 



otherwise 



u , 

-^phydyas(/) = ^ ^ ^fc 



otherwise 



k=-(K-l) 



MKsin 



77\ sin(7rt/'ig) cos(7rat/'i g) 

P TC V) ~ 7Tt/T s l-l0L'*t'*/T'£ 



0<T CP , 
< T w 



K 



Band-limited 



RC 



RRC 



Half sine 



Half cosine 



(T s , 



Prc(f) 



\f\< 



1 + COS 



;^(m-^f)) 



l-OL 

2T S 



I — a <- I f I <- l+o: 
2T S —Ml— 2T S 

otherwise 



(l-a T +4^ 



Prrc(t) = < 



£ = 



otherwise 



sin ((l- Qr )^.J+^* cos ((1+ Qr )^± ] 



Prrc(/) = V / ficT7y 



RRC (q r = Of 



RRC (q r = Tf 



< q r < 1 



< q r < 1 



Localized 



Gaussian 



Hermite 



IOTA 



EGF 



Pgaussian(t) = (2a g ) 1 / 4 e~ 7ra g t (/) = Pgaussianfo V^g) 



nite , hermite 



V>n,hermite (*) = #n fv^Ft) e"^ , if n (t) = (-l) n e* 2 -j^e"* 2 



||x(t — fca) || 



, x(t) e E 



f (*) = I 



^ dk,a,v [pgaussian (t + k/vQ,a) + Pgaussian (* ~ fc^O,**) 



$^ d U/a,T cos(27T^/r ) 

1 = 



< a K 



7V H 



< 



tipliers and calculus of variations. Note that since only even 
indexed prolate functions are even-symmetrical, only d2k are 
considered through the optimization procedure. By applying 
similar optimization procedure, OFDP has been utilized in 
other studies (9). Also, another optimization procedure which 
is based on deriving the composite Nyquist response of OFDP 
filter instead OFDP itself is suggested in [|46l to reduce the 
optimization complexity in |44| . 



4 ) Kaiser Filter: Another efficient solution for a filter with 
finite length is proposed by Jim Kaiser by employing Bessel 
Functions to achieve an approximation to the PSWFs B71 . 
(48). Basically, it offers a suboptimal solution for the out- 
of-band leakage. A favorable property of Kaiser filter is its 
flexibility to control the sidelobes and stop-band attenuation, 
through a single design parameter f3. The expression is given 
in TABLE M where N is the filter order, I (x) = 1 + 

v^oo \ {x/2) k ~ " 



denotes the modified zeroth order Bessel 
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TABLE III: PHYDYAS filter coefficients for different over- 
lapping factors 11501 . 





K = 3 


K = 4 


K = 6 


K = 8 


ao 


1 


1 


1 


1 


ai 


-0.91143783 


-0.97195983 


-0.99722723 


-0.99988389 


a 2 


+0.41143783 


+0.70710681 


+0.94136732 


+0.99315513 


CL3 




-0.23514695 


-0.70710681 


-0.92708081 


CL4 






+0.3373834 


+0.70710681 


CL5 






-0.07441672 


-0.37486154 


CLQ 








+0.11680273 


CL 7 








-0.01523841 



function. To achieve a minimum stop-band attenuation of 
a s h dB, value of f3 may be selected as follows |[49l 

P = 

'0.1102(a sb -8.7) , a sh > 50 , 

< 0.5842(a sb -21) - 4 + 0.07886(a sb -21) , 21 < a sh < 50 , 
, a sh < 21 . 

(14) 

Moreover, choosing N w 14 ^gX/ t 5 — yields a transition band 

Of A/tms. 

5j PHYDYAS Filter: A time-domain filter with good fre- 
quency localization has been specified in l50l where the 
coefficients ctk for different filter lengths are specified in 
TABLE [Till The frequency response of this filter is given 
by ED. PHYDYAS filter has been investigated further in [|T0l . 
and was then subsequently adopted to be used in the European 
PHYDYAS project on FBMC fTOt I5D. 153. 

B. Band-limited Filters 

These filters are theoretically defined with finite bandwidth 
support. However, they theoretically require infinite filter 
length in time domain. Therefore, these filter are problematic 
considering practical slot and frame formation. 

1 ) Raised Cosine Filter: Raised-cosine filter is a commonly 
used Nyquist filter in which the time-frequency localization is 
controlled by a parameter that is referred as the roll-off factor 
(a r ). While raised cosine filter results in rectangular shape in 
frequency domain for a r = 0, the filter shape becomes half 
part of the cosine function when a r = 1. The filter bandwidth 
is calculated by (1 + a Y )/T without any filter truncation. 

2) Root-Raised Cosine Filter: Typically, RRC filter used 
at the transmitter and receiver to satisfy Nyquist criterion is 
derived from raised-cosine filter. If the frequency response of 
RRC filter is given by P rrc (f), then, the relation P rc (f) = 

(f)Prrc(f) is obtained. 

3) Half Cosine and Half Sine Filters: RRC filter with 
a r = 1 yields half cosine filter (HCF), which is given by E), 
0, CD, E3, EH. The half cosine filter provides a good 
compromise for time/frequency behavior; its relaxed transition 
bands allow approximation through a relatively short time- 
domain filter, while achieving high attenuation in the stop- 
band ifTA . On the other hand, RRC filter with a r = is 
called as half sine pulse. While half sine results in minimal 
bandwidth, it is very susceptible to the filter truncation. 



C Localized Filters 

In contrast to the finite support in one domain as in time- 
limited and band-limited filters, localized filters consider both 
domains on an equal footing. Therefore, they try to provide 
the compactness in both time and frequency domains. They 
can give the isotropic (same) responses in time and frequency 
domains. They tend to behave as a Gaussian pulse because of 
its maximum compactness. 

1) Gaussian Filter: Gaussian filter is given by |[T2lL lf54l- 
l57ll with a control parameter a g . Using the properties of 
Fourier transform, one may show that the frequency response 
of a Gaussian function is also another Gaussian function fT2ll . 
If a g = 1, Gaussian filter is perfectly isotropic. On the other 
hand, since p gaU ssian(^) > for all t, Gaussian filter does 
not satisfy Nyquist criterion. In other words, it causes ICI/ISI 
from a certain symbol to its neighboring lattice grid points in 
Fig. El 

2 ) Hermite Pulses: Hermite functions have been commonly 
used in the literature to obtain well-localized pulse shapes 
in doubly dispersive channels fljj, El, |20|, E2i EH- 
They are obtained from Hermite polynomial functions H n (t) 
of different orders n which can span the space Hilbert 
space £ 2 (R) of square-integrable functions. Note that Hermite 
pulses of different orders are orthogonal to each other with 
respect to the weight function e _7r£ . Therefore, by deforming 
the Gaussian filter with higher order Hermite functions, zero 
crossings in time are provided to satisfy Nyquist criterion. The 
general filter design approach based on Hermite functions is 
to linearly combine weighted versions of Hermite pulses with 
different orders which provide isotropic dispersion in time and 
frequency domains [ 59 1 in order to obtain a prototype filter 
that will perform best for a given channel environment [HTL 
l20ll . 1571 . Also, employing the orthogonality between Hermite 
functions, concentric toroidal pulses are introduced to increase 
the spectral efficiency of the transmission [|58l . 

3) Isotropic Orthogonal Transform Algorithm Filter: The 
isotropic orthogonal transform algorithm (IOTA) filter targets 
to preserve the excellent localization property of Gaussian 
prototype filter, and orthogonalizes it to prevent ISI and ICI to 
neighboring symbols in time/frequency lattice. Starting with 
the Gaussian function, orthogonalized pulse is obtained as 
shown in TABLE [TT] where T and T~ x are the operators 
for Fourier transform and its inverse, respectively, ||-|| is L 2 - 
norm, and O a is an orthogonalization operator on a function. 
The resulting prototype filter fulfills the Nyquist criterion. For 
a = 1, ro = V2T/2 and vq = y/2F/2, pi t&(t) becomes 
identical to its Fourier transform, and therefore becomes an 
isotropic filter. With these parameters, IOTA filter corresponds 
to optimal Nyquist filter in terms of time-frequency localiza- 
tion when the rectangular lattice is considered. [|32l . 

4) Extended Gaussian Function: Extended Gaussian func- 
tion has a closed-form expression given by [60| with tqVq = 
0.5, 0.528^o < a < 7.568^o > an d real valued coefficients 
dk,a,v can be computed (for finite number of coefficients bkj) 
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as (6B 

d k , atV0 = £ b Kj e~^ 2 ^ 2 ^ ,0<k<K, (15) 

3=0 

where [61 ] lists the coefficients bkj for < k < 14 and 
< j < 7. Note that similar to the relation in Gaussian 
function, EGF has the same shape with its Fourier transform, 
except a scaling term ll62l . Two special cases of EGF are as 
follows 11601: 

. IOTA: For a = 1, r = V2T/2 and is = V2F/2, EGF 
becomes the IOTA filter. 

• TFL1: When EGF is truncated (e.g., into one sym- 
bol duration - and hence the naming TFL1), modula- 
tion/demodulation delay may be shortened, and good 
time-frequency localization can be still maintained 
through some optimization procedures ll63l-[|66l. 

IV. Tools for Prototype Filter Analyses 

In order to better understand the impact of prototype filters 
in FBMC schemes, various tools and metrics are introduced 
in this section. Use of these tools will be instrumental for 
assessing the performances of different prototype filters. 

A. Time-Frequency Localization 

It basically states that a narrow waveform yields a wide 
spectrum or a wide waveform results in a narrow spectrum, 
and making narrow in both domains is not possible. Time- 
frequency localization of a filter is measured by Heisenberg 
uncertainty parameter £mter which is given by 

, _ \W)\\ 2 <1 n „ 



where 




t) 2 \p(t)\ 2 dt 




f) 2 \p(f)\ 2 df 



(17) 



(18) 



a t is the time dispersion (or the standard deviation of the pulse 
in time), and 07 is the frequency dispersion (or the standard 
deviation of the pulse in frequency), and i and / are arbitrary 
real number!] El, El, 1521 . 

B. Direction Parameter 

The direction parameter k G [0, 00), which shows how a 
pulse shape lies in time-frequency plane, is given in [|54l as 

k - — 

For example, while k is equal to for the rectangular filter, 
Gaussian filter with a g = 1 gives k = 1 because of its 
isotropic dispersion. Larger k means a pulse stretched more 
along the time axis compared to the frequency axis. 

2 t and / are not necessarily the mean values of the pulse in time and 
frequency, respectively. 



(19) 



C. Ambiguity Function 

Generalized Nyquist criterion, discussed in more detail 
in 0, (El, E3, G3, Hi, (6B, states that the projection 
of transmitter and receiver prototype filters should be zero at 
every integer multiple of symbol spacing in both time and 
frequency as 

(Pmk(t),Pnl(t)} = 

Ptx (t ~ mro) p r * x (t - nr ) e'^-^dt = 5 mn 5 lk . 

(20) 

By using fractional values instead of using integer multiples, 
equation given in (l20l) also leads to a more generalized form, 
known as the ambiguity function ifTTl . 



As* (0,^) 



(21) 



The properties of ambiguity function are given below: 

• Ambiguity function is a two dimensional correlation 
function in the time-frequency plane. 

• Ambiguity function yields real values in the case of an 
even- symmetric prototype filters. 

• Generalized Nyquist criterion can also be expressed in 
terms of the ambiguity function as 



Asr (nr > ) 



n = l = 
otherwise 



(22) 



Ambiguity function characterizes the amount of interfer- 
ence that leaks into other symbols in the time-frequency 
grid. Ambiguity function gives an intuitive demonstration 
of ICI/ISI robustness. It indicates the sensitivity to time 
and frequency impairments. 

Similar to time-frequency localization of a filter, ambigu- 
ity function also cannot be concentrated arbitrarily . The 
limitation is given by l32ll 



^ambiguity 



\\PtAt)\\ 2 \\PrAtW 

27RT0<X0 



< 1 



(23) 



where 




>) 2 |4(^,^)| 2 dW , (24) 



(25) 



and, cj) and ip are arbitrary real numbers. 

In the literature, ambiguity function is also used to obtain 
other functions. For example, interference function which is 
defined by 



(26) 



might provide better visualization on ICI and ISI lfT2l . (60). 
In order to simplify the expressions of average SIR due to 
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dispersion in the channel, (0, if;) is changed by introducing 
additional phase shift p as 



1) SIR for OFDM/QAM: For OFDM/QAM, p mk (t) and 
p n i(t) can be expressed as 



=e 



-j2np 



p tx (t-0)p r * x (t)e^Mt , (27) 



^W=Ptx(t-mr )e^ o£ , 
^W=Prx(t-nr )e^ o£ , 



where A (0, -0) is the modified ambiguity function. As^ (0, ^) 
can be derived as (0, ^) A* (0, ^) | p= ^ 0/2 - 

Z). Orthogonality Parameter 

The reconstruction quality of the prototype filter which is 
investigated by the orthogonality parameter of a\ [54 1 given 
by 



(34) 
(35) 



respectively, where tqVq = 1 as in Fig. [3 |[a)| Then, assuming 
the impact of ICI and ISI on each subcarrier is statistically 
equal to each other and all subcarriers are utilized, H n imk 
can be written as 



l T \ v 



H(t, u)x 



-■{ 



X n \ — X n i 



(28) 



mr - r) p r * x (t) e j2 ^ kVQ+v)t dtdvdr . 

(36) 

Therefore, selecting p = kv§r, (l36t is simplified to 



Using 
as 



it is possible to write the average SIR performance H mk = j f H(r, v)A (mr + r, kv + ^)d^dr . (37) 

J T J V 



SIR 



2 ' 

I 



a 



(29) 



If X m k are independent identically distributed with zero mean 
and (ptx (t) ,Prx (t)) = 1, the power of desired part and the 
power of interference from other symbols are obtained as 



where dg and of are the power of the desired part and the 
interference leaking from other symbols, respectively. SIR, 
as defined here, should ideally be infinity, since no other 
interference sources are considered. However, orthogonality 
can be still spoiled by not only due to the lack of exact 
representation of the filter in digital domain, but also due to the 
doubly dispersive channels. In the literature, statistical charac- 
teristics of the channel are generally described with wide- sense 
stationary uncorrected scattering (WSSUS) assumption [11 3011 
given by 

E{#(r,z/)} = , (32) 
EiH^^H^n,^)} = S H (r.^diT-r^iu-^) , (33) 

where Sh (t, v) is the channel scattering function. The term of 
wide sense stationary ensures the autocorrelation is stationary. 
In other words, statical characteristics of the channel do not 
change in time, and are only related with time difference as 
in d33i The term of uncorrelated scattering shows that one 
of the delay components of the received signal is uncorrelated 
with all the other delay components. However, note that the 
WSSUS assumption ignores the non- stationary characteristics 
due to the distance dependent path loss, shadowing, delay drift, 
and the correlation between the reflected rays from the same 
physical objects 11311 . In addition, WSSUS assumption is not 
valid for short-term channel characteristics, especially, when 
the specular reflections dominate over diffuse scattering [ 132]. 
Yet, WSSUS assumption is widely used in the system models 
to characterize the wireless channels because of its simplic- 
ity. For example, exponential decaying multipath with Jakes 
Doppler spectrum [11331 or ITU models [11341 for different 
environments are commonly used for the channel scattering 
function for terrestrial communications. Considering WSSUS 
assumption, analytical expressions of cr| and of also can be 
obtained for OFDM/QAM and OFDM/OQAM as follows: 



4 = E {|X 00 tfoo| 2 } = J J S H (r, v) \A (r, ^)| 2 d^dr 



{ 



of = E < Xqo — XoqH { 



00 



(38) 



^ / S h (t,v)\A (ttito + r, kv + v) \ 2 &v&t , 



(ra,fc)^(0,0) 

respectively, where E{-} is the expected value operator. 
The illustration of 



(39) 



is given in Fig. 1 31a) [ where the 
absolute value of the ambiguity function at the grid points 
(black circles) are equal to zero in single-tap channel with 
no Doppler spread. Due to multipath and Doppler spread 
(captured by the illustrated scattering function), the desired 
symbol (red circle) observes ISI/ICI from neighboring grid 
points. 

2) SIR for FBMC/OQAM: For FBMC/OQAM, which is a 
variation of FBMC, Pmkif) and p n i(t) can be expressed as 

Pmk(t) = j m+k Pt* (t - mr ) e^ ot , (40) 
Pni(t) = j n+l Pr, (t - nr ) e?™ 1 ** , (41) 

respectively, where tqVq = 1/2. Using (l40t and (|4TT) , © can 
be expressed as 



■H-rnk — H r 



Imk 



1=0 
n=0 



■m-\-k 



I I H(t, v)A (raro + r, kvo + ^)d^dr . 

J r J V 



(42) 



Therefore, a\ and of are obtained as in (l43b and (l44l) , respec- 
tively, where ^{-} expresses the imaginary part of argument. 
By calculating 5ft{A (0, ip)} and S{A (0, ip)}, the impact of 
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-2t -t -c JC^2t 

(a) OFDM/QAM; ICI/ISI from carriers (b) OFDM/OQAM; ICI/ISI from real carriers (c) OFDM/OQAM; ICI/ISI from imaginary carriers 

Fig. 5: SIR Calculations for OFDM/QAM and OFDM/OQAM. 



a!=E{|^{X 00 #oo}| 2 }= f [ S H (T,is)\U{A(T,is)}\ 2 disdT , 

J T J V 

a\ =e||x 00 -X 00 i^oo| 2 ] = Yl [ [ SH(r,iy)^^j m+k A(mro+r,kiy + iy)}\ 2 diydr 



(43) 



{m,k}^{0,0} ' 



XI / / S h ( t ' ") l K M ( mT ° + T ' fc ^o + v)}\ 2 AvAt + ^ f f Sh(t, v) |3 {A (mr + r, fc^ + i/)}| 2 di/dr 



(m,fc)^(0,0) 
m+fcGEven 



(m,fc)^(0,0) * 
m+fceOdd 



(44) 



the interference from real (i.e. mod {m + /c, 2} = 0) and 
imaginary (i.e. mod {m + /c, 2} = 1) carriers in a doubly 
dispersive channel environment can be visualized on time- 
frequency grid as in Fig. I 3[b)| and Fig. | g[c)| 

V. Evaluating Prototype Filters 

In this section, comparisons of the prototype filters de- 
scribed in Section [TTTI are presented. First, time-frequency 
characteristics of the filters and the power contribution to the 
other symbols in frequency domain are given in Fig. [6] Then, 
the spectral leakage performances of the filters in multicarrier 
scheme are given in Fig. | 3Ia)| and Fig. I 31b)| for 5 MHz and 
20 MHz bandwidth utilizations in LTE. While 300 and 1200 
subcarriers are utilized for v§ = F, proportionally, 424 and 
1698 subcarriers are used for = F/y/2. The ambiguity 
surfaces of the filters are provided in Fig. 0to visualize both 
ICI and ISI. Considering both OQAM and QAM, SIR in 
(l29t plots are given in Fig. [9] for different prototype filters. 
In addition, the impact of truncation on SIR is investigated 
for band-limited and localized pulses, which can be seen in 
Fig. [TO] In all other figures, K is fixed to 16 for the sake 
of implementation purposes unless otherwise stated. Also, the 
energies of the filters are normalized to 1 . 

Time-frequency lattice is adjusted considering the stag- 
gering strategy of in-phase and quadrature components. For 
example, symbols are placed to the lattice where tq = T 
and u = F for rectangular filter in non- staggered struc- 
tures. Additionally, rectangular filter with CP and windowing 
(Xcp = Tw = 0.14T) is included to compare the conventional 
OFDM with FBMC with the other filters. On the other hand, 



while ro and v$ are set to V2T/2 and V2F/2, for Gaussian, 
Hermite, and IOTA filters (a g = 1), r = V2T/2 and 
vq = y/2F/2, respectively, the lattice where ro = T/2 and 
= F is constructed for RRC (a r = 0.22), half sine (RRC 
where a r = 0), half cosine (RRC where a Y = 1), prolate, and 
OFDP. 

In Fig. [9l SIR plots are obtained by taking four channel 
scattering models into account; 1) Carrier frequency offset, 2) 
Timing offset, 3) ITU Vehicular A model with Jake's Doppler 
spreading, and 4) ITU Pedestrian A model with Jake's Doppler 
spreading. For ITU channel models, modified Jake's Doppler 
spectrum, which also includes the time variation of the phase 
components of the reflected radio waves B135I . is applied in 
order to prohibit the infinite density at the spectrum. In this 
model, correlation between phase components is defined with 
B parameter which is a positive constant. In the simulations, 
heuristically, B is set to 0.5 fa where /a is the maximum 
Doppler shift. 

A. Time-limited Filters 

Rectangular prototype filter corresponds to sine function in 
frequency domain as given in Fig. [6] Therefore, OFDM has 
high sidelobes as given in Fig.|3Ia)|and Fig. QIb)! It is possible 
to suppress the sidelobes of OFDM by applying proper win- 
dowing. However, the spectral leakage of the rectangular pulse 
is still considerable compared to the other filters. Note that 
the sidelobes are naturally suppressed in FBMC approaches 
without sacrificing the spectral efficiency. 

In terms of SIR, extended rectangular filter, which corre- 
sponds to conventional OFDM, provides the best performance 
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(d) Time-limited filters (K > 1). (e) Spectral leakage of time-limited filters. (f) Power contribution to other symbols. 




for the less frequency dispersive channel scenarios as in 
Fig. [9] because it is more robust to time dispersion at the 
expense of CP overhead. Employing CP extends the null 
in ambiguity surface in time domain as given in Fig. [ g[b)[ 



However, FBMC approaches might be more robust to strong 
frequency dispersion depending on the filter used, such as 
Hermite, IOTA, OFDP, and PHYDYAS filters in Fig. Ha)l 
Fig.BTc)! and Fig.^d)) Note that when CP is not used, OFDM 
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Fig. 7: Comparison of the power spectral densities of FBMC with different prototype functions considering LTE subcarriers 
utilization. Energy of each subcarrier is normalized to 1. 



loses its advantages in multipath environments. Additionally, 
Fig. [9] shows that since OQAM allows more room for time- 
frequency spreading of prototype filters [IT2l . OFDM/OQAM 
provides superior performance than OFDM/QAM when the 
same prototype filter is employed (e.g. rectangular/QAM ver- 
sus rectangular/OQAM). 

Time and frequency responses of the PHYDYAS, prolate, 
and OFDPs are given in Fig. [ 0d)[ Since their durations are 
longer than the rectangular filter, they provide better spectral 
leakage performances compared to the rectangular filters as 
can be seen in Fig.[0e)[ Especially, prolate filter provides very 
sharp decaying in frequency domain by discarding the Nyquist 
criterion. However, if the Nyquist criterion is provided, the 
out-of-band power is increasing as in OFDPs compared to the 
prolate filters. Also, providing the Nyquist criterion in both 
domains increases the sidelobes of the OFDPs. PHYDYAS 
filter provides very good out-of-band leakage characteristics. 
Note that Nyquist criterion in time does not hold when the time 
shift is more than 2 for PHYDYAS filter (K = 4) as given 
in Fig. [ g[c)[ This issue yields better sidelobe performance 
compared to the OFDPs. Additionally, the out-of-band leakage 
performances of these filters in the system level are given in 
Fig- 1 3I a )l an d Fig. | 3Ib)[ They are also superior than conven- 
tional windowing approaches without degrading the spectral 
efficiency of the system. 

The ambiguity surface of prolate filter given in Fig. l g{d)| is 
similar to the ambiguity surface of the Gaussian pulse given 
in Fig. However, it provides compact support for a given 
filter length in frequency domain instead of both time and 
frequency domains. Since OFDPs are derived from the proper 
combinations of prolate sequences, they are also compact in 
frequency domain while satisfying the Nyquist criterion. If the 
Nyquist criterion is provided only in time domain, OFDP gives 
minimal ICI contributions to the other symbols in frequency 
domain for a given filter length. Thus, exploiting OFDPs 
for FMT-based FBMC systems is more suitable compared to 



RRC based filters, since there is no degradation on Nyquist 
criterion for OFDP because of the filter truncation. Indeed, if 
the Nyquist criterion is enforced in frequency domain, OFDP 
characteristics approach the rectangular pulse with longer filter 
length as shown in Fig. [ ^f)[ Therefore one can interpret that 
OFDPs stays between RRC and rectangular pulse. 

Both PHYDYAS filter and OFDPs have remarkably good 
SIR performances in Fig. [9] after Hermite and IOTA filters. 
Considering the SIR performances and their the finite dura- 
tions, which corresponds to less complex systems, these filters 
are promising for FBMC based systems. 

B. Band-limited Filters 

As opposed to the time-limited, band-limited filters, i.e., 
RRC family given in Fig.[ ffig)| are ideally limited in frequency 
domain. However, in practical conditions, the power spectrum 
of these filters will deviate from ideal frequency localization 
as shown in Fig. [ 0h)| because of the filter truncation in time 
domain. While employing larger a r (e.g., half cosine) results 
in larger bandwidth, using small a Y (e.g., half sine) apparently 
increases the sidelobes of FBMC for the same filter length as 
given in Fig. |3Ia)|and Fig. U>)[ The impact of truncation can 
be clearly observable by using ambiguity surface for filters 
with small a r factors. For example, half sine does not satisfy 
the Nyquist criterion when K = 16 and the nulls for symbol 
spacing do not occur as in Fig. [ g[i)[ In Fig. [lOl the impact of 
the truncation is given in terms of SIR. Apparently, for lower 
a r , the impact of truncation on SIR performance is highly 
dominant. For example, SIR is limited to 20.5 dB for K = 16 
for half-sinc filter even if the channel dispersion is ignored. In 
111201 . it is also mentioned that the orthogonality loss due to the 
shortly truncated RRC filters is compensated via equalization. 

the frequency responses are almost flat in the passband 
region, which means that the per- subchannel equalizers also 
improved the frequency response of t 
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In all plots in Fig. [51 half-cosine filter approaches to 
PHYDYAS and OFDP in terms of SIR performances since 
these filters are very similar to each other. However, decreasing 
a r degrades the SIR performance of RRC, since the impact 
of the filter truncation on SIR becomes dominant due to the 
increasing sidelobes in time domain. Additionally, RRC and 



half-cosine filters approximately have the same immunity to 
timing misalignment as given in Fig. [ ^Ib)[ 

C. Localized Filters 

While time-limited and band-limited pulses have tendency 
to localize the energy in a single domain, localized filters 
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Fig. 9: SIR performance of the prototype filters for different channel scattering functions 



provide a compact support in both time and frequency do- 
mains. Therefore, the localized filters have isotropic responses 
in both time and frequency characteristics as in Fig. 
and Fig. [ 0k) [ The isotropic also corresponds to isotropic 
ambiguity surfaces as given in Fig.[g[|j)| Fig.[£ p:)[ and Fig. [01)1 
Gaussian filter provides a circular ambiguity surface without 
any nulls. However, the localization of Gaussian filter is the 
best compared to the other filters and decays fast in both time 
and frequency domain. Thus, in terms of out-of-band leakage, 
Gaussian filter yields better sidelobe performance as given in 
Fig- I 31a)[ and Fig. I 3[b)| In addition, the performance of the 
Gaussian filter is almost constant compared to the other filters 
as can be seen in Fig. [9] Even if Gaussian filter has worse 
SIR results, the required effort for equalization for FBMC with 
Gaussian filter might be less than other prototype filters since 
Gaussian pulse has the best time-frequency localization. 

On the other hand, by slightly degrading the time-frequency 
localization performance of the Gaussian filter, Hermite and 



IOTA achieve Nyquist criterion for To = V2T/2 and = 
V2F/2 when a g = 1. Therefore, the nulls are located for 
every V2F and V2T as given in Fig. [ 01)[ In terms of out- 
of-band leakage, FBMC with Hermite filter provides superior 
frequency response compared to the IOTA filter as given in 
Fig- 1 3fa)| and Fig. Qtb)[ When both SIR performance and out- 
of-power leakage are considered, Hermite and IOTA filters 
have the better characteristics because of two fundamental 
reasons: 1) very good joint time-frequency localization and 
2) being orthogonal filters. Both of them provide very strong 
time-frequency dispersion immunity as well as CFO and 
timing misalignment robustness as given in Fig. [ 2[a)| and 
Fig. 1 2tb)[ Additionally, the impact of filter truncations on SIR 
is not as dominant as other filters. For example, Hermite and 
IOTA filters with K = 8 is sufficient to yield more than 
80 dB SIR as in Fig. \W[ On the other hand, these filters lose 
their advantages for low Doppler scenarios compared with the 
extended rectangular, i.e. conventional OFDM. According to 
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Fig. and Fig. EJd)) while FBMC with Hermite and IOTA 
filters are better than conventional OFDM for fa > 135 Hz 
for ITU Vehicular A, they provide better characteristics for 
frequencies as low as fa = 15 Hz for ITU Pedestrian A. 

VI. Transceiver Design Issues 

This section addresses several issues that have not been 
directly addressed in the earlier sections and that are related to 
implementation of FBMC systems. Representative references 
are provided to other related work in the literature and some 
of the key insights are highlighted. For a more comprehensive 
treatment and a larger body of literature related to these 
problems, readers are referred to corresponding references. 

TABLE [IV] provides a classification of various references 
in the literature related to prototype filter design. This table 
should be treated as a set of representative references, and 
an attempt to tentatively capture and classify a large body 
of literature that are relevant to FBMC systems. In each 
column of TABLE HVl commonly used prototype filters in 
the FBMC literature are listed, while each row of the table 
includes a specific aspect of FBMC systems. References for 
different prior work on FBMC are placed in the table based 
on the prototype filters used in each reference, as well as the 
scope of the corresponding reference. The goal is to capture a 
tentative research activity map of this area, which is useful 
for understanding the key contributions and open research 
directions. 

A. Lattice and Filter Adaptations 

In a doubly-dispersive channel, the lattice and filter param- 
eters that minimize ISI and ICI are given by 
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pulse adaptation lattice adaptation 

where r rms is the root mean square (RMS) delay spread of 
the channel and fa is the maximum Doppler shift. While 



lattice adaptation corresponds to aligning tq and with r rms 
and /d max , pulse adaptation is equivalent to the dilation of 
the pulse depending on the channel dispersion in time and 
frequency. The rationale behind these adaptations are to match 
the proportion of the channel dispersion to the pulse dispersion 
in the time and frequency. 

In an early study, H), the pulse adaptation is given with- 
out theoretical explanations. In a later work, the identity of 
eft/erf = r max //d max is obtained theoretically in [ 136] when 
r max /d max — » 0. This identity is utilized in lfT6l , ll88l consid- 
ering underspread channels and improved in [ETL [|32l , [|60l , 
ll86l considering the lattice adaptation. For example, in ETTl . 
both pulse adaptation and lattice adaption are jointly optimized 
to adapt to the doubly-dispersive channel characteristics based 
on WSSUS assumption. Considering rectangular lattice and 
Gaussian filter, the optimum conditions for the pulse and 
lattice adaptations are derived as 
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where (l45t corresponds to a doubly dispersive channel with a 
uniform delay power profile and uniform Doppler power spec- 
trum, while (l46l) corresponds to a doubly dispersive channel 
with an exponential delay power profile and U- shape Doppler 
power spectrum. 

In l32l . a theoretical framework to adapt the pulse shape 
and lattice to varying channel conditions is introduced. It 
is emphasized that OFDM with a rectangular lattice is a 
suboptimal solution for doubly dispersive channels. Instead of 
rectangular symbol placement, if the lattice is constructed with 
a hexagonal shape, i.e. neighboring symbols located at each 
corner of hexagon, better protection against ISI and ICI com- 
pared to rectangular lattices is achieved since the minimum 
distance between symbols increases. For this purpose, a lattice 
generator matrix A is utilized to determine the frequency 
spacing and time spacing as 
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where p > lfT2l . While the choice of p = yields rectangular 
lattice, p = 0.5 results in hexagonal structure. Note that both 
lattice structures provide the same spectral efficiency which is 
calculated by r] = /3/det {A}. While the normalized minimum 
distance between symbols is 1 for rectangular lattice, it is ob- 
tained as Vl-25 for hexagonal lattice. Therefore, the immunity 
of the system against time-frequency dispersion is increased 
without losing from the spectral efficiency. For exponential 
decaying and Jake's Doppler channel model, the improvement 
in signal-to-interference-plus-noise ratio (SINR) is in the range 
of 1-2 dB in case of forcing the Gaussian filter to be a 
Nyquist filter with maximum spectral efficiency. Additionally, 
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considering dilation and chirp operations, the pulse adaptation 
is combined with the hexagonal lattice structures. In (86), 
Lattice-OFDM is re-investigated with nonorthogonal pulses 
by emphasizing the fact that the orthogonalization procedure 
destroys the time-frequency concentration of the initial pulse. 
Instead of the maximizing number of symbols per second per 
hertz via the orthogonal pulses, orthogonality is abandoned 
with Weyl-Heisenberg frames similar to [21], and matching 
equations for the pulse and lattice adaptations are provided 
for the hexagonal lattice structure. 

Note that it is not an easy task to develop a single lattice 
structure and a prototype filter for a system serving multiple 
users with different channel characteristics. To deal with 
the lattice and pulse design for such a system, while one 
might consider the worst case scenario to optimize pulse 
and lattice, other might include multiple lattices and filter 
structures within the frame. In H1371 . unlike the conventional 
orthogonal frequency division multiple accessing (OFDMA) 
frame structures that consider the worst case communication 
channel, multiple CP and subcarrier spacings are employed 
by taking the statistics of the mobility and the range of 
the users into account. As a result, better frequency spread 
immunity and higher spectral efficiency are obtained. Even 
if the proposed method does not apply pulse adaptation, 
intuitively, it offers lattice adaptation for multiple users. 

B. Equalization 

One of the features of FBMC transmission different from 
OFDM is the absence of CP. It is well-known that using 
CP longer than the maximum excess delay of the channel 
in OFDM allows single tap frequency domain equalization. 
FBMC follows a different rationale than OFDM by limiting 
the numbers of the symbols contributing to desired symbol 
in time and frequency domains and by exploiting good time- 
frequency localization properties of the filters. Hence, FBMC 
offers manageable ICI and ISI characteristics even in doubly 
dispersive channels, at the expense of relatively more complex 
equalization. Thus, considering the equalization complexity at 
the receiver is critically important for the sake of practical 
systems. 

In fHTll and fT38l . equalization methods for FBMC trans- 
mission are introduced under three categories. The first ap- 
proach is to use well-localized prototype filters in order to 
deal with the time- frequency selectivity. The pulse energy is 
concentrated in time and frequency domains and the effects 
of ICI and ISI on the neighboring symbols are assumed 
to be limited. In this context, a basic single-tap subcarrier 
equalization per subcarrier might be employed to recover 
the symbols (64), 11381 . [139]. However, since single-tap 
equalizers use the inverse of the channel response at center 
frequency of each subcarrier, the performance of the channel 
estimation directly affects the equalization performance. The 
second (conventional) approach exploits fractional sampling 
to equalize the FBMC symbol via finite impulse response 
(FIR) filters. It is natural for the equalizer to work at the 
fractional sampling, e.g., T/2, since the FBMC symbols are 
generated with staggered modulations. In this category, it is 



possible to develop equalizers considering different criteria. 
For example, minimum mean square error (MMSE) equaliz- 
ers 111231 . 111381 . l!14Ql -[ 1421. maximum likelihood sequence 
estimator (MLSE) equalizer, [143), low-complexity equalizers 
(H, (ED, Ifl38l and channel tracking equalizers J95), JT20ll . 
11441 all have different optimization goals. The third approach 
utilizes multiple taps via FIR filters for each subcarrier. The 
equalizer operates at the symbol rate and deals with the 
aliasing between the subcarriers for FBMC subcarriers that 
arise due to the staggered modulations [121] . Theoretically, 
it is possible to perform equalization at the symbol rate 
by exploiting equalizers which consider multiple subcarriers, 
but it does not bring any specific advantage compared to 
fractionally spaced equalizers [138]. 

C. MIMO 

FBMC has been considered for wireless standards such as 
3 GPP in the past. However, one of the most critical challenges 
has been how to implement MIMO with FBMC, causing 
skepticism for further standardization studies. FMT, which is 
one of the FBMC variations as discussed in previous sections, 
can actually be conveniently used along with MIMO l59ll . 
On the other hand, spectral efficiency of FMT is lower than 
that of OFDM/OQAM due to larger separation between the 
subchannels. 

In 11161 . it was shown that MIMO with spatial multi- 
plexing can be directly applied OFDM/OQAM when the 
MMSE equalization is utilized 111 1411 . Ill 1811 . Use of spatial 
multiplexing with maximum likelihood detection (MLD) or 
the Alamouti space time block coding (STBC) with FBMC 
remain to be challenging problems due to the signaling model 
of OFDM/OQAM E3, lfTT4ll . In particular, data is carried 
only by the real part of the signal in OFDM/OQAM, and 
imaginary part appears as an interference at the receiver [11 171 . 
11181 . Therefore, MIMO techniques that are easy to implement 
in CP-OFDM such as STBC are no longer possible with 
OFDM/OQAM, and imaginary interference needs to be han- 
dled. While 11151 proposes a method to cancel this imaginary 
interference for the case of single-delay space-time trellis 
coding (STTC) with two transmit and one receive antennas, 
11171 extends this to N t transmit antennas and proposes an 
iterative decoding approach. 

In 11181 . Alamouti coding was shown to be applicable to 
OFDM/OQAM when it is further combined with code division 
multiple access (CDMA). Recently, 111411 introduced a new 
scheme referred as FFT-FBMC to remove the intrinsic interfer- 
ence, which yields a system model similar to OFDM. Hence, 
MIMO transmission schemes that are possible in OFDM, such 
as spatial multiplexing with MLD or Alamouti STBC, can also 
be utilized with FBMC. All in all, there still needs to be more 
studies on the application of MIMO to FBMC under practical 
implementation scenarios. 

D. Cognitive Radio and Resource/Power Allocation 

Due to its favorable properties, OFDM has been commonly 
considered as a candidate physical layer technology for dy- 
namic spectrum access and cognitive radio systems [11451 - 
11471 . FFT part of the OFDM demodulator can also be 
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conveniently utilized for spectrum sensing purposes to identify 
the presence of primary users in the vicinity [l79l . [11481 . On 
the other hand, OFDM has its own drawbacks when used in 
cognitive radio applications. For example, sidelobes of the 
OFDM subcarriers may cause large interference leakage to 
neighboring channels. This prevents efficient utilization of 
unused portion of the spectrum by secondary users and hence 
limits the aggregate spectral efficiency of a cognitive radio 
system. Moreover, OFDM is highly sensitive to time/frequency 
synchronization errors, which puts stringent constraints for 
dynamic spectrum access in cognitive radio networks. 

Low spectral leakage property of FBMC makes it an 
attractive candidate for cognitive radio systems [|79l . While 
a subchannel in CMT or SMT overlaps with its immediate 
adjacent subchannels only, FMT variation of FBMC utilizes 
non-overlapping subchannels. As such, due to the minimiza- 
tion of spectral leakage to neighboring subchannels in FBMC, 
secondary users in a cognitive radio network may efficiently 
utilize the spectrum opportunities. Moreover, with FBMC, 
no tight synchronization is required between primary and 
secondary networks, which relaxes implementation complex- 
ity. Finally, f79ll shows that the use of filter banks yields 
similar spectrum sensing accuracies when compared to those 
of optimum multi-taper based spectrum sensing [146], with 
less computational complexity. 

Downlink resource allocation in FBMC-based cognitive 
radios is investigated in [77]. The goal is to maximize total ca- 
pacity of the cognitive radio network, under certain constraints 
such as the total available power and interference threshold for 
primary users. Results show that an FBMC-based cognitive 
radio network yields higher aggregate spectral efficiency with 
significant gains under certain scenarios (up to 200% in some 
settings), and yields lower interference to the primary users. 

In H101I . uplink resource allocation in a multiple antenna 
cognitive radio system has been investigated. In this case, 
FBMC yields aggregate spectral efficiencies that are around 
45% higher than those delivered by OFDM. A similar con- 
clusion regarding uplink spectral efficiency has been reached 
in 111061 . while [[72l shows through simulations that FBMC 
provides superior BERs when compared to OFDM in a mul- 
tiuser uplink channel, at a much lower computational com- 
plexity. There are two main reasons for the spectral efficiency 
gain in both the downlink and the uplink when FBMC is used. 
Firstly, as opposed to OFDM, FBMC does not allocate any 
time (and hence, energy) to the transmission of a CR Secondly, 
FBMC yields lower adjacent channel interference; this may 
improve the SINR of secondary users, and/or allow the use of 
smaller guard bands among subchannels. 

In |4Q| . a multi-mode uplink access method is proposed. 
In this approach, depending on the specific requirements of 
different users, different access methods such as SC-FDMA, 
FBMC, or FB-S-FBMC can be utilized by different users. 
Such a hybrid approach provides full flexibility in cognitive 
radio applications. 

E. Time/Frequency Synchronization 

Sensitivity of FBMC systems into time/frequency synchro- 
nization errors are investigated in [52] and HI 001 through 



simulations, and in ll90l through theoretical analysis. In [[52l 
and 11001 . 2-D contour plots for which a given combination 
of CFO and fractional time delay (FTD) yields a 3 dB degra- 
dation in a target SINR are simulated for WiMAX systems. 
Results show that FBMC is less sensitive to CFOs compared 
to OFDM. On the other hand, in the presence of FTD, larger 
degradation observed in FBMC due to absence of a CR 

Handling combined effects of FTD and CFO in FBMC 
systems is a challenging problem, and requires advanced signal 
processing techniques in FBMC transceivers [|90l . Theoretical 
analysis in [[90l shows that FMT, at a cost of lower spectral 
efficiency when compared with SMT and CMT, is most robust 
against synchronization errors, especially at high SNR. For the 
uplink, FMT and OFDM/OQAM are particularly effective to 
combat against the degradation caused by the misalignments 
among the transmissions of different users (see also [60], [|72ll ). 

Time and frequency synchronization for FBMC systems 
have been addressed in ED, ES, lfT09Tl — ifTTTTl . Since there 
is no CP in FBMC, CP-based timing synchronization is not 
possible with FBMC as opposed to OFDM. While preamble- 
based, training-based, or blind synchronization methods as 
in OFDM are applicable, some modifications are required. 
In particular, presence of complementary interference from 
neighboring symbols (in time and frequency) necessitates the 
design of pilots that can work well with FBMC [109]. One 
approach that yields good performance is to use pilots and 
auxiliary pilots H1081 . 111091 . H1241 . Alternatively, training se- 
quence based ll92l and non-data aided (blind) synchronization 
techniques are also available in the literature H1491 . H1501 . 

F Channel Estimation 

As also mentioned, orthogonality of OFDM/OQAM holds 
only in the real field. Therefore, conventional channel esti- 
mation techniques for OFDM cannot be used without any 
modification for OFDM/OQAM, due to the intrinsic imagi- 
nary interference which is removed via operation l63l . 
ifTOl . lfT25l . lfT38l . (ED- To solve this problem, pilot-aided 
channel estimation techniques are proposed in [124]. In [|63l . 
an alternative preamble-based channel estimation approach 
named interference approximation method (IAM) by using 
real- valued symbols is introduced. It does not require a priori 
knowledge of the prototype function, and yields 1 dB gain 
in the BER performance when compared with CP-OFDM. In 
111041 . the real- valued symbols are replaced with imaginary 
ones and 2 dB gain is achieved compared to CP-OFDM. Then, 
a general theoretical framework for IAM preamble design is 
given in [125]. Also, a comparative study on the preamble- 
based least square (LS) channel estimation is given for OQAM 
and QAM systems considering sparse preambles in [11511 . 

G. 60 GHz Communications 

Due to large bandwidths available, 60 GHz communications 
carries critical importance for next-generation broadband com- 
munication systems. Design of analog components is one of 
the important challenges at such high central frequencies, and 
may lead to radio-frequency (RF) front-end non-idealities such 
as phase noise (PN) and CFOs. OFDM, while coping well 
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(a) Synthesizing single FBMC symbol without ID FT. 
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(b) Synthesizing single FBMC symbol with ID FT, which generates the 
same output as in Fig. 11(a). 
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(c) Combining multiple FBMC symbols. 

Fig. 11: Synthesizing FBMC symbols with IDFT operation. Note that X m k is the information symbol on mth time slot and 
kth subcarrier, [pi,p2, ~-,Pkn] are the samples of the prototype filter, and Fi are the polyphase components of the prototype 
filter. 



with the high frequency selectivity that is typical for 60 GHz 
communications, is known to be very susceptible to PN and 
CFO. In |[T07l . robustness of FBMC against the following 
non-idealities of 60 GHz communications are investigated 
and compared with continuous phase modulation (CPM): 1) 
high PN of the local oscillator at mixing stage, 2) low- 
power (and hence, low-resolution) requirement of analog-to- 
digital converter (ADC) due to high bit rates, and 3) distortion 
and spectral regrowth caused by non-linearities in the power 
amplifiers (PAs). While FBMC/OQAM has better spectral 
efficiency than CPM, results in |107| show that CPM is less 
sensitive to PN and ADC resolution imperfections, and allows 
more power efficient operation of the PA compared to FBMC. 

H. Impact of Non-linear Power Amplifiers and PAPR Issues 

The PAPR characteristics of OFDM and FBMC are com- 
pared in KB, El, EH. The PAPR of OFDM and FBMC are 
shown to be similar for various scenarios in [70 1, [78], with 
FBMC providing lower spectral sidelobes in its power spectral 
density. Similarly, [4(3 shows that FBMC and OFDM have 
practically identical PAPR cumulative distribution functions 
(CDFs). On the other hand, use of DFT-spreading go), |)6) 
and filter bank spreading BUI in conjunction with FBMC 
lowers the PAPR of the transmitted signals, yielding PAPRs 
that are close to those of SC-FDMA. 

Impact of linear and non-linear PAs on the power spectral 



densities (PSDs) of OFDM and FBMC are modeled and com- 
pared in l78l . While FBMC has significantly lower sidelobes 
than those of OFDM when the PA operates in the linear region, 
the benefit of FBMC diminishes when the PA operates in the 
non-linear region. Since the user equipments closer to the 
cell edge have to transmit with higher power levels due to 
power control. Therefore, they may have to operate in the non- 
linear region of the PAs and benefit of using FBMC might be 
incremental for such users. 

/. Polyphase Implementation 

FBMC introduces complexity by adding extra filters at 
the transmitter and the receiver. However, the complexity of 
FBMC can be reduced by exploiting polyphase representa- 
tions of filters and FFT operations. As shown in Fig. [T] the 
polyphase network is after the FFT at the transmitter, and 
before the IFFT at the receiver, and they replace the CP 
insertion/removal blocks in OFDM. An illustrative scenario for 
analyzing and synthesizing FBMC symbols with 3 subcarriers 
are shown in Fig. Qj] and Fig.[l2l respectively, where both PPN 
and DFT implementations are described. It is assumed that the 
filter length is 4 N where N = 3. Thus, the filter is effective 
for 12 samples for each FBMC symbol. First, single FBMC 
symbol is constructed in Fig. II Ufa)] After the prototype filter is 
multiplied with an information symbol in each branch, FBMC 
symbol is generated by combining the branches modulated 
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Applying PPN decomposition. 
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2) Removing unnecessary samples 



Analyzing kth subcarrier along with PPN implementation. 
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(b) Considering all subcarriers with DFT operation. 
Fig. 12: Analyzing FBMC symbols with DFT operation. 
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. In Fig. [Hb)| 

an equivalent diagram of Fig. Ill|ra)| is realized with inverse 
discrete Fourier transformation (IDFT) operation by applying 
the multipliers indicated in Fig. Ill|ra)| after the summation 
operation. To make these substitutions, the repetition of expo- 
nential term W$ for every other three samples is exploited. In 
Fig. Ill|rc)[ three FBMC symbols are taken into consideration. 
Each FBMC symbol is shifted by symbol spacing, which 
is 3, and added to generate an FBMC frame. Indeed, this 
operation corresponds to a pattern as indicated in Fig. Ill|fc)| 
In this example, operations involving all red boxes indicated 
on the figure correspond to PPN component Fq. Essentially, 
the output of each IDFT result is filtered with some filter 
coefficients. Therefore, FBMC brings some additional filtering 
operations after IDFT operation. Note that these filters are 
simply equal to 1 for OFDM. 

The basic idea behind the FBMC receiver consists of three 
basic steps; 1) Shifting the desired subcarrier to baseband. 
2) Filtering the signal with receiving filter to eliminate the 
impacts of other subcarriers. 3) Sampling at correct instants. 
Considering these steps, analysis of a single subcarrier is given 
in Fig. 1 121(a)] First, polyphase decomposition is applied to filter 
using shift operation (z~ x ). Since the information symbols 
are constructed in every other 3 samples and the exponential 
term repeat itself around the unit circle, exponential terms 
can be distributed to each phase as factors. Also, since the 
intermediate samples are not necessary, the implementation 
is simplified using a commutator and removing the zero 



coefficients at each branch. Then, by moving the positions 
of the exponential coefficients before summation operations 
and analyzing each subchannel, DFT operations is obtained 



as in Fig. I12|rb)l Similar to the transmitter, additional filtering 
operation is necessary before DFT operations. 

The implementations given in Fig. [TT] and Fig. [12] are 
suitable for non- staggered and staggered modulations. For 
staggered modulations, the structures have to be replicated 
for quadrature and in-phase part separately and combined 
with proper time shift to yield complete FBMC frames. In 
addition, DFT and IDFT operations should be centralized in 
time domain to implement even- symmetrical filters. Extended 
summaries on PPN are also provided in J7], (49), H152I . 
Additionally, PPN structure is implemented and discussed in 

a m, ma, m, ma, una. 

/. Testbeds and Extensions to Standards 

While there are many publications theoretically comparing 
FBMC with OFDM and other waveforms, there are very 
limited number of works that discuss testbed implementation 
of FBMC are relatively limited. In ifTOOl . an FBMC testbed, 
which developed as a part of the PHYDYAS project E3, HI 5411 
is described. The testbed is capable of real-time transmission 
and reception of FBMC signals. Transmitter side involves a 
field-programmable gate array (FPGA) that includes the PHY 
layer only, and can operate both in OFDM and in FBMC 
modes. The receiver side is composed of an RF front-end, 
a USRP motherboard (involving ADCs and FPGA), and a 
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software subsystem. 

Recently, there are also several studies that investigate 
the extension of FBMC technology into existing wireless 
standards. For example, FBMC has been considered in the 
context of WiMAX systems in El, OH, EH, lHH, and 
in the context of beyond 3G systems in ll65lh 1661 , [105 ]. Both 
l65l and l66ll propose a modification in LTE frame structure so 
that it may be used together with FBMC. In particular, absence 
of a CP in FBMC enables the transmission of an additional 
OQAM symbol within one slot duration composed of seven 
complex OFDM symbols in LTE. While this extra OQAM 
symbol may also be utilized for other purposes, it has been 
used to improve the channel estimation quality in l65l . l66l . 
In order to better assess the various implementation trade-offs 
between OFDM and FBMC, further studies are needed on the 
testbed experimentation. 

K. Complexity Analysis 

One of the important criteria for the adoption of FBMC in 
the future wireless standards is whether it can yield sufficiently 
better gains than existing techniques such as OFDM, at the 
cost of a reasonable complexity increase. The complexity of 
FBMC has been investigated in the literature from different 
perspectives in @, £CD, EU, ED, El, El, EQ-E1, 
11551 . As discussed in ll52ll . the major factor for complexity 
increase in FBMC is due to the replacement of IFFT/FFT in 
OFDM with the filterbanks. 

On the other hand, additional complexity has to be intro- 
duced into OFDM transmitter in order to suppress out-of- 
band power leakage, which is handled naturally by FBMC. 
A typical approach is to use active interference cancellation 
(AIC) in OFDM, which reserves a number of subcarriers at 
the edge of an OFDM band to suppress sidelobes in the PSD 
of OFDM [68], ED, El- Implementation of complex equal- 
ization and MIMO techniques in FBMC may also increase its 
complexity further, see e.g. [81], [|83l . 

VII. Concluding Remarks and Discussion 

Waveform design is one of critical issues for future radio 
access technologies. Recently, filter bank based multicarrier 
schemes, re-emerging as an umbrella for the conventionally 
employed schemes, stand as an exciting and promising con- 
cept. In this study, the waveform design issues are examined 
under the filter bank based multicarrier transmission and re- 
ception techniques. Starting with Gabor systems, a framework 
on the prototype filter design is provided considering both 
orthogonality and biorthogonality properties. We emphasize 
that while having lower lattice density provides better SIR 
performance in doubly dispersive channels, SIR performance 
can be maximized with the optimum biorthogonal filters by 
extending the concept of orthogonality to biorthogonality. In 
addition, conventional FBMC approaches, i.e., SMT and CMT, 
provide high equalization complexity. Therefore, the impact 
of biorthogonal filters to the equalization domain is described. 
Many Gabor systems consider the design of optimum filters 
that maximize the SIR performance. In this paper, we further 
emphasize the need for Gabor systems which also consider the 



receiver equalization complexity to combat with the doubly 
dispersive channel characteristics. 

In this study, the various performances of the well-known 
prototype filters in literature are investigated considering dif- 
ferent tools and metrics. According to the results, even if 
OFDM benefits from CP utilization yielding simple frequency 
domain equalization for low Doppler spreads, it loses its 
advantage for higher frequency dispersion. Hermite and IOTA 
filters achieves higher SIR for the environments with higher 
frequency dispersion, and intrinsically provide better sidelobe 
characteristics. On the other hand, PHYDYAS filter and OFDP 
achieve superior SIR performances with finite durations which 
allow less complex structures at the receiver and transmitters. 
Also, compared to the other prototype filters, RRC with higher 
a r , Hermite, and IOTA filters are robust to the filter truncation. 

It is important to emphasize that filters introduce different 
characteristics and trade-offs to the schemes. Therefore, it 
is not an easy task to develop a single Gabor system that 
addresses every aspects of the systems. With the current 
technologies, three main aspect cannot be ignored in the 
design of the accessing schemes: simplification of the equal- 
ization, MIMO functionality, and applicability to the network. 
OFDM addresses the simplification of the equalizers and 
MIMO functionality with the exploitation of CP. However, 
efficient utilization of OFDM is still a discussion considering 
heterogeneous and denser networks. The heterogeneous and 
denser networks are the promising wireless approaches, since 
they allow systems to operate at the same spectra. Recently, 
significant efforts have been devoted to maximize the spectral 
efficiency and network throughput within novel networking 
strategies. At that point, FBMC approaches provide a perspec- 
tive to examine the schemes in terms of filter that also includes 
these networking scenarios. Beside these facts, exploiting other 
degree of freedoms such as having multiple users in the system 
can be considered as another open research area. 
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